Focusing on the issue of speech recognition in multi-noise environment, a new hierarchical speech recognition model considering environmental noise as the context of speech recognition was proposed. The proposed model was composed of two layers of noisy speech classification model and acoustic model under specific noise environment. The difference between training data and test data was reduced by noisy speech classification model, which eliminated the limitation of noise stability required in feature space research and solved the disadvantage of low recognition rate caused by traditional multi-type training under certain noise environment. Furthermore, a Deep Neural Network (DNN) was used for modeling of acoustic model, which could further enhance the ability of acoustic model to distinguish noise and speech, and the noise robustness of speech recognition in model space was improved. In the experiment, the proposed model was compared with the benchmark model obtained by multi-type training. The experimental results show that, the proposed hierarchical speech recognition model has relatively reduced the Word Error Rate (WER) by 20.3% compared with the traditional benchmark model. The proposed hierarchical speech recognition model is helpful to enhance the noise robustness of speech recognition.